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Abstract. At the time of the current global 
energy crisis energy awareness education 
in school is increasingly important. 
Textbooks directly helping and influencing 
learning play an important role in this, 
especially in the case of natural sciences. 
Thus, the aim of this research was to show 
how science textbooks help to educate 
energy awareness (EA). Regarding this, 

67 Hungarian science textbooks were 
analysed using a quantitative textbook 
analysis from grade 1 to 12. When studying 
the knowledge on EA, the occurrence of 
the key concepts related to EA included 

in the National Core Curriculum (2020) 
was examined according to content, 
pedagogical and structural dimensions. 
According to the results, the most 
common key concepts were renewable 
energy, and energy saving, while the 

least common was the concept of energy 
crisis. The textbooks studied increasingly 
communicate key concepts in the form of 
definitions in the main textbook text at the 
level of information, moving towards the 
upper grades, and paying less attention to 
the attitude-forming elements. There are 
few tasks that require the application of 
knowledge about EA, especially those that 
also develop digital competence. In this 
form, the textbooks studied are less suited 
to developing an attitude toward EA, which 
does not help in making the right decisions 
in relation to EA. 
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Introduction 


Today's global energy crisis increasingly requires the energy aware- 
ness (EA) of the members of society. To do this, it is not enough to know the 
concepts and knowledge associated with EA, it is necessary to develop an 
attitude in society that will enable us to act decisively in everyday life and 
make responsible decisions in the field of energy use. This process of educa- 
tion must begin in early childhood, and it must be achieved that knowledge 
expressed in words leads to actual behaviour, not as proved by several studies 
(Coker et al., 2010; Fehér & Revakné, 2021; Kénya, 2018a; Revak, 2019; Tayci & 
Usual, 2012; Tortop, 2011), that we say one thing and do the other. EA educa- 
tion is a joint task of education and society, through which the knowledge, 
behaviour and emotional attitude must form a complete whole in the rising 
generation. In addition to the knowledge gained at school, the behavioural 
patterns and examples that our students see outside the walls of the school, 
at home, in the family, and in their immediate and more distant environment 
are decisive factors. In school, the basic organizational form is the teaching 
class, where elements of environmental education, including EA education, 
appear either directly or indirectly, in most cases integrated into science 
subjects. Its content is determined in Hungary in the National Curriculum, 
which appears partly in textbooks and partly in the environmental education 
programme of schools as a general educational goal and task. These environ- 
mental education programmes determine what activities, programmes and 
educational tasks are implemented in the school during classes and outside 
the classroom. Thus, a student's attitude and knowledge in the field of EA is 
the result of a system of multiple effects that interact closely with each other 
(Larson, 2010; Weaver 2002), of which this study shows the content and peda- 
gogical elements of textbooks, including science textbooks, which play an 
important role in the learning process, contribute to EA education. Although 
the study covered Hungarian textbooks the applied textbook analysis method 
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has an international significance since the present study focuses not only on the content related to EA but also on 
the pedagogical and structural elements important in EA education. These elements together facilitate a higher 
level of EA based on the efficient application of textbooks in learning. In order to understand where the textbook 
analysis in the present study occupies a place in the system of textbook research, it is necessary to cover some 
of the research methods and results of the concept and evaluation of textbooks, and also those of the textbook 
analysis of environmental education, including EA education. 


Textbook Concepts 


The definition of textbooks is usually based on their functional role. According to Pingel (2010), textbooks 
convey age-appropriate disciplinary knowledge to students in an understandable way to help them learn. G6mez 
etal. (2014) have also highlighted the learning function of textbooks, while others have emphasized that textbooks 
represent the curricula that communicate knowledge, and develop abilities and skills (Kim & Pang, 2022; Rezat et 
al., 2021). In addition to the requirements set by the curricula, cultural and ideological content must also appear 
in textbooks, which together with disciplinary knowledge and elements that help learning and the development 
of abilities, ensure that students are closer to reality and everyday life (Rotger, 1997). However, all this can only be 
realised with the help and the teaching methods of teachers, who can interpret and encode these textbook texts 
with their expertise and pedagogical skills (Caravita et al, 2008; Zahorik, 1991). Based on the different textbook 
concepts, textbooks are educational aids that, based on the curriculum requirements, help the learning process and 
the personality development of the students with the help of various elements of content, form, and pedagogy. 


Textbook Evaluation Models and Methods 


The first demands for textbook analysis appeared in the 19th century. Until the end of the 1970s, textbook 
research was characterised by textbook revision, textbook reconciliation, and textbook correction, devoid of a 
scientific standard. Textbook research from the seventies to the present day has had a multilateral approach, and 
in addition to the content elements, the format, didactic, pedagogical, social, and scientific characteristics of text- 
books have also been examined (Dardai, 1999). However, even today, these analyses lack the impact assessments 
of textbooks on learning, abilities, and personality development. 

Some of the multilateral approaches have sought totality. In historical order, such a model was the “Bielefeld 
raster’, which was described in 1986 by three professors of the University of Bielefeld, who already clearly considered 
textbook analysis to be a scientific activity. The five dimensions studied by them (metamemory, textbook design, 
branch of science, specialised didactics, education science) and the 24 categories within them were referred to as 
one large examination raster. This model was groundbreaking in its own time, as it also investigated the function 
of metamemory, which plays an important role in the learning process. It was a pedagogically very well-recognized 
textbook analysis model, which was, however, not often used in practice (Dardai, 1999; Kojanitz, 2007). 

The textbook analysis model of Weinbrenner (1995) has also aimed for totality and has considered that text- 
books can be examined in five dimensions by longitudinal (historical) and cross-sectional (comparative) analyses 
using acontent analysis procedure. The dimensions include scientific theory, specialist science, scientific didactics, 
educational science, and design (Dardai, 1999). Weinbrenner’s model has a great similarity to the “Bielefeld raster’, 
with the difference that it does not examine aspects of metamemory but seeks scientific theory analysis instead. 
Thus, the model focuses less on learning as a process. 

Compared to the multilateral models mentioned here, the textbook evaluation model, based on the peda- 
gogical trend that interprets learning as a constructive student activity, represented progress. This gave textbook 
analyses a global, systemic framework, which analysed the content and pedagogical details; the main aspect is to 
help students think and make learning more effective. The primary representative of the constructive approach 
is the work of Chambliss and Calfee (1998), which has formulated three main approaches to textbook production 
and evaluation: comprehensibility, exemplary curriculum, and student-centred instruction. According to their 
proposal, the examination of the content, pedagogy, and structural sections of textbook analysis should also be 
subordinated to these three aspects (Dardai, 1999). This model is still used in textbook analysis research to this day, 
providing a framework for the evaluation of textbooks. According to this framework, the primary consideration 
in the content, pedagogical and structural analysis is whether it helps or hinders the learning process, knowledge 
acquisition and personality development of students. 
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Recent textbook research today, following the textbook analysis model based on constructive student activ- 
ity, mainly consists of the analysis of concepts in the material, as well as how these concepts are integrated into 
the textual and non-textual elements, what is the visual presentation of the content to be learned, and how the 
didactic processing of the curriculum helps learning (Dardai, 1999). 

Vojir and Rusek (2019) in their systematic review, have grouped publications on science textbook analysis 
published between 2018 and 2020, based on evaluation criteria. In the list they have collected, the subjects of 
studies with the learning process in mind have been the following: “relationship with curricula, learning, social 
relations of the natural sciences, the historical development of textbooks, e-textbooks and open textbooks, errors 
and misconceptions in textbooks, comparison of textbooks from different geographical areas, textbooks from 
teachers’and students’ point of view (Yilmaz et al., 2017). Among the studies, an increasing number of analyses can 
be found focusing on the forms, in which sustainability and climate change occur in textbooks (Roman & Busch, 
2016; Simsek, 2011). 

The concepts and methods of content and structure analysis can often be encountered in the textbook litera- 
ture. According to Weber (1990), the content analysis procedure as a research method uses a series of procedures 
to draw conclusions from the text. Examples of such methods are coding schemes and their testing, computerized 
text classification, the creation of reliability and classification schemes, assumed and inferred categories, alterna- 
tive classification systems, and single or multiple classifications. Weber’s method of content analysis suggests both 
qualitative and quantitative methods of analysis, as does a much earlier concept of content analysis that originated 
from Berelson (1952). According to it, content analysis means structuring, categorisation, and qualitative and quan- 
titative evaluation according to specific objective criteria. The majority of research that performs content analysis, 
like those mentioned here, categorizes according to specific criteria and looks for the frequency and place of a 
given category in the text (Devetak & Vogrinc, 2013; Qadeer, 2013). 

There are usually three levels of structure analysis: general structure, text analysis, and visual content analysis. 
When evaluating the general structure, they include page numbering for the chapters, the size of the chapters, the 
ratio of text and illustrations, etc. In the course of text analysis, it is studied whether there is a fragment of the text 
that encourages teacher explanation, teacher demonstration, student assignments, experimentation, problem- 
solving, or the book contains a section on repetition and summary in the chapters, a collection of concepts at the 
end of the book. For example, the structural analysis of the visual content of the book discusses the presence and 
proportion of photos, drawings, graphics, tables, the ratio of visual elements and text fragments (Abubaker & Lu, 
2011; Devetak & Vogrinc, 2013). 

It can be seen that both content and structural analysis imply the possibility of qualitative and quantitative 
evaluation, both separately or in mixed form. Qualitative evaluation is an analytical method with much more 
subjective elements than quantitative evaluation, which relies more on the analyst's own value system and un- 
derstanding of the text. In the course of this method, it is essential to accurately define the categories that are the 
subject of the analysis (e.g., what aspects of the history of science are in the textbook, the analysis of the scientific 
correctness of the concepts, what elements of the formation or maintenance of the type of misconception can be 
observed, what types of errors occur) (Pingel, 2010). 

In the course of quantitative analysis, the frequency of the textbook element under consideration and/or the 
magnitude and extent of its position in the text are most often examined (Pingel, 2010). For this analysis, one, two 
or more scientists can perform the coding of the studied elements. Hardly any examples of coding comparison can 
be found in the literature of textbook evaluation, most of the time it is not even mentioned how many people did 
the coding. For example, the publication of Kim and Pang (2022) on the analysis of the content on sustainability 
occurring in primary school mathematics textbooks mentioned that coding was performed by two encoders and 
Interclass Correlation Coefficient (ICC) was used to test code matching. 

The two evaluation methods have been used simultaneously by the vast majority of textbook analysts, as the 
two methods together give a more complete picture of the content, structure, pedagogical characteristics of the 
textbook being studied (Kim & Pang, 2022). 


Research Studying the Occurrence of Environmental Education and Energy Awareness in Textbooks 
Environmental education content appeared in textbooks as early as the end of the 18th century, however, 


their textbook study intensified only in recent decades. For example, under the 1992 PHARE (Poland and Hungary 
Assistance for Restructuring their Economies) programme, 201 books with environmental content have been 
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analysed. The results have revealed that the approach to environmental education hardly appeared in textbooks. 
Children rarely encountered with sustainability. The study has also shown that teachers did not take advantage of 
the specific methods of environmental education, but rather traditional teacher-centred methods (e.g., teacher 
performance, explanation, discussion) were typical. 

Later research has mainly focused on the analysis of the environmental education aspects of science textbooks 
(Kénya, 2018b; Kojanitz, 2003; Oguz et al., 2004; Salmani et al., 2015). One of the largest programmes was BIOHEAD- 
CITIZEN (Biology, Health and Environmental Education for Better Citizenship) 2006. This programme has brought 
together 19 countries to examine areas of environmental and health education and biology education. The aim 
of the research has been to formulate recommendations based on the results for improving teaching on biology, 
health, and environmental topics and for policymakers. The analyses have had a special focus on the examination of 
illustrations in the textbooks and the analysis of the curriculum according to whether it was linearly or non-linearly 
constructed (Horvath et al. 2009). An important conclusion of the programme has been that environmental topics 
were covered in a small number of pages in the books, which would be necessary to increase in order to acquire 
appropriate knowledge and even more so to shape the environmental attitude. The low level of illustrations related 
to environmental topics has also been cited as a problem, which has been a major shortcoming in understanding 
environmental topics and problems, motivating and raising awareness for these topics. In other words, environ- 
mental education through textbooks is more of a knowledge than an attitude-forming nature (Kénya, 2018b). 
Further textbook analyses linked to different countries can be found in the international literature, which studies 
the environmental education content of textbooks and the presence of ecological knowledge in them. Kojanitz 
(2003) for example has examined the extent to which geography books in Hungarian vocational schools met the 
requirements of environmental education. In this textbook analysis, the statistical, data-like recording of the en- 
vironmental aspects of the textbooks has been performed. Kojanitz (2003) counted the lines that were related to 
environmental science obtaining in this way a quantitative indicator that allowed the determination of the ratio of 
environmental aspects to the total textbook content. Salmani et al. (2015) conducted a content analysis of newly 
prepared Iranian social and natural science textbooks. The study has covered the topics of ecology, environmental 
protection, and human activity. The analysis has focused on both content and structural elements, its unit was the 
textbook page itself, with text, images, questions, and exercises. The results have shown that in both social and 
natural science textbooks, the topic of ecology was dealt with to the greatest extent, followed by human activity 
and then environmental protection. 

Most of the research on EA education has studied the knowledge and attitude of students related to renewable 
energy. Coker et al. (2010) measured the knowledge of Turkish primary and secondary school students on renewable 
energy, which showed that students had many misconceptions regarding renewable energy. The content of the 
studied science textbooks related to renewable energy is also responsible for these misconceptions. Knowledge is 
communicated, in most cases, in short definitions that do not describe the context. Tortop also conducted studies 
with Turkish high school students in 2011. The research has shown that students did not have the right knowledge 
and attitude towards renewable energy sources, they had more misinterpretations, and they did not have accurate 
knowledge, for which textbooks could have also been blamed. 

The number of textbook content analyses focusing on multiple aspects related to EA is small in the interna- 
tional literature. In Hungary, Uténé-Kiss, (2012) and Revakné et al. (2018) have studied the content, structural, and 
pedagogical characteristics of energy awareness education in natural science and geography textbooks in primary 
and secondary schools. The main conclusion of their study has been that in the analysed textbooks, EA education 
was much more knowledge-like, with little content and structure related to attitude shaping. Similar results have 
been obtained by Chao et al. (2017), who have studied the knowledge, attitude, and behaviour of high school 
students learning from two different geography textbooks in relation to energy saving and carbon reduction. The 
results have revealed that the proportion of knowledge elements in the geography textbooks was much greater 
than that of attitude and behaviour. The study has applied content analysis by searching key concepts related to 
energy saving and carbon reduction and analysing their ratio related to the text of the whole textbook. 


Research Problem and Focus 
The basic problem studied by the present research, based on previous research results, is that the energy 


awareness of students is lower compared to factual knowledge level, which can be explained by the fact that, apart 
from other influencing factors, textbooks (primarily in natural sciences) do not emphasise EA education enough 
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in content, pedagogy and formal aspects (Coker et al., 2010; Fehér & Revakné, 2021; Konya, 2018a; Revak, 2019; 
Tayci & Usual, 2012; Tortop, 2011). 
Regarding the above problem, content analyses were made previously in relation to the project entitled “The 
Role of social learning processes regarding renewable energy resources in the Eastern Region of Hungary” (sup- 
ported by the NKFIH, K 116595 application in Hungary, 2016-2019). These, however, included only natural history 
(grades 1-6) and geography (grades 7-9) textbooks. The aim of the research is to continue this textbook analysis 
after 2019 and extend it to every textbook in natural sciences distributed in Hungary. 
The questions of the research were the following: 
1a) What is the proportion of knowledge related to EA specified in the National Curriculum (2020) in the 
studied science textbooks? 
1b) Secondly, in analysing the professional content dimension, the description and determination of 
knowledge related to EA and the distribution of knowledge according to the determination methods 
are examined. 
2a) Analysing the pedagogy dimension of the textbooks, the first question is what didactic elements are 
used in deepening and applying knowledge related to EA in the studied textbooks? What are the ratios 
of the key concepts among the didactic elements? 
2b) Analysing the pedagogy dimension of the textbooks the second question is what is the ratio of EA-related 
attitude development in the studied textbooks and what is the ratio of occurrence of the examined 
knowledge in these parts? 
3) Analysing the structural dimension of the textbooks the question is what structural units are typical in 
the studied textbooks and what is the ratio of EA-related knowledge among them? 


Research Methodology 
General Background 


The theoretical framework of this textbook analysis was provided by a textbook evaluation model that was 
based on a pedagogical trend that interprets learning as a constructive student activity. In this way, the textbook 
analysis focuses on how textbooks help the learning process and the shaping of the way of thinking of students 
(Chambliss & Calfee, 1998; Vojir & Rusek, 2019). In the course of the research, the content that meets the EA require- 
ments of the 2012 and 2020 National Curricula, including the appearance and definition of knowledge and concepts 
of EA, was analysed in the spirit of comprehensibility and straightforward curriculum planning. The occurrence of 
knowledge and concepts related to EA was also subjected to structural and pedagogical evaluation, considering 
how much these help students to learn and develop their attitude towards EA. The evaluation was carried out 
using a quantitative content (conceptual) analysis method, evaluating content related to EA from professional, 
pedagogical, and structural aspects. 


Sample 


The present research was carried out between 2018 and 2021, during which currently available science 
textbooks written for grades 1-12 were analysed in Hungary in relation to EA education. The selection was based 
on the officially licensed and most commonly used textbooks in public education included in the List of public 
education textbooks 2018/2019 (Educational Authority, 2018). Thus, 67 science textbooks from five publishers 
distributing them were analysed. Students study natural history in grades 1-6, biology in grades 7-12, chemistry 
in 7-10, geography also in grades 7-10, and physics in grades 7-12, therefore the studied textbooks were analysed 
in the distribution of the given subjects according to these grades (see Table 1). The content of the textbooks is 
based on the requirements of the National Curriculum (2012 & 2020) for the given subject and for environmental 
and EA. Among the textbooks, four groups of textbooks were found (two in biology, one in geography, and one 
in natural history), where the same author wrote textbooks for different grades (even three or four grades) of the 
subject in question. In other cases, one author wrote textbooks for one or two grades within a given subject. 
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Table 1 


The Number of Studied Textbooks (n) According to Grades and Subjects 


Grade 
Subject ——— a 
1 2 3 4 5 6 7 8 9 10 11 12 
Natural history (n) 4 4 3 1 4 4 - - - - - - 20 
Biology (n) - - - - - - 2 2 2 3 2 2 13 
Chemistry (n) - - - - - - 2 2 2 3 9 
Geography (n) - - - - - - 3 5 3 3 14 
Physics (n) - - - - - - 1 2 2 2 2 2 11 
> (n) 4 4 3 1 4 4 8 1 9 1 4 4 67 
Procedures 


In the course of the study, after formulating the goals and questions of the research, the theoretical framework 
of the textbook analysis was determined as a first step. The National Curriculum (2012 & 2020) is fundamentally 
built on the constructive pedagogical direction when designing the requirements. Therefore, the multidisciplinary 
textbook evaluation model that emphasises the learning process as an active activity as the basis for analysis was 
also considered (Chambliss & Calfee, 1998; Vojir & Rusek, 2019). Based on this, professional content, structural and 
pedagogical analysis dimensions were studied, bearing in mind in each case how the studied dimensional ele- 
ment contributed to the learning process (see Figure 1). The method applied in the study was quantitative content 
analysis and conceptual analysis within it. The occurrence frequency of key concepts related to EA was analysed 
along professional, pedagogical and structural dimensions. 


Figure 1 
Outline of the Research 


Textbook analysis methods preferring a 
constructive learning procedure 


Analysis from professional aspect Analysis from pedagogical aspect Analysis from structural 
a) Searching for knowledge related a) Applying knowledge related to EA aspect 
to energy awareness (EA) b) Shaping attitude Knowledge related to EA 
b) Type of description, definition in structural units 


In the course of conceptual content analysis from a professional aspect, first, the findability of knowledge re- 
lated to EA in textbooks was examined. The search was based on the EA requirements of the National Curriculum 
(2012 & 2020), which defined what key concepts related to EA should be discussed during the learning of natural 
sciences within the framework of environmental education (in Hungary, the elements of environmental education 
appear mainly integrated into natural science subjects.). In order to make the scope of knowledge relevant to EA 
specific and well-defined, the following key concepts were searched: renewable energy, energy saving, heating, 
power station, conscious energy use, energy crisis, energy efficiency, and energy consumption. The presence of 
the particular key concept in a given textbook was examined. It was repeatedly observed that it was not the key 
concept itself that appeared in the textbook, but some knowledge about it. For example, it was not the concept 
‘renewable energy’ that appeared, but wind or hydropower in the atmosphere or hydrosphere topics. In this case, 
it was considered that the concept of renewable energy appeared figuratively more than once in the textbook, 
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i.e. in it the occurrence rate of the given key concept was greater than in a textbook where the given key concept 
was found in only one case (e.g. a definition) The occurrence of each key concept in the entire textbook sample 
was studied, and then occurrences were examined by grades and subjects as well. 

Those cases when no knowledge related to EA could be found in the studied textbook were eliminated from 
the data set. Thus, a data set without cases with zero occurrences was used for the assessment. In the course of 
the content analysis, the way in which the key concepts were described and defined was examined again. Two 
aspects seemed reasonable, either definition-like or indirect specification of the concepts. This categorization was 
carried out because, according to previous studies (Adamik et al. 2004; Nagy Varga, 2014) the process of acquiring 
knowledge and understanding processes could be influenced by the fact that the given concept was present in 
textbooks in the form of a short, specific, to the point definition or longer text with explanatory elements, in a 
circumscribed way. How this concept definition influences the factual knowledge of students could be the subject 
of future research. The ratio of key concepts occurring as definitions and circumscriptions in the total sample, as 
well as by grade and subject. In analysing the occurrence of the key concepts according to these two aspects, 
as well as when examining them in additional dimensions, the co-occurrence of all key concepts and not single 
concepts was examined based on specific criteria. (Examining each key concept would be the subject of separate 
studies. Here we would like to give a general picture with a focus on EA). With regard to the key concept occur- 
rences according to the way they are described and defined, the fact whether the concept can be found in one 
or both (both: definition and circumscription as well) definition types in the given textbook was looked at. One 
of the options for acquiring and recording knowledge more deeply in textbooks is the appearance of the given 
knowledge in several content or didactic contexts. 

As didactic aspects and elements of the pedagogical dimension, three variables were analysed: textbook 
tasks, homework tasks and tasks that require internet application. The cognitive, social, etc. abilities these tasks 
develop were not studied. This may be the subject of a different research. The occurrence of key concepts for the 
total number of cases was examined in this case as well, which was the fewest here. This is the result of the fact 
that even if a concept occurred in the textbook, the task for its application may not occur in the textbook. Thus, this 
further reduced the number of cases that can be investigated. Analyses by both grade and subject were carried 
out, as well as co-occurrence studies (whether the key concept occurred in one, two or all three types of tasks). 

In the course of the pedagogy analysis, the analysis of the distribution of the key concepts was the next 
according to the aspects of attitude shaping, with the aim of seeing the extent to which the studied textbooks 
contributed to the development of the attitude related to EA. Two aspects seemed logical for the analysis. One was 
the textbook elements of attitude shaping itself (textbook sections on behaviour, emotion, belief, and attitude) 
and the other was the occurrence of the given key concept in elements that relate only to its factual knowledge 
acquisition. The total number of occurrences of the key concepts actually found in the textbooks, as well as their 
occurrence by grade, subject and co-occurrence, according to the textbook sections that provide attitude-shaping 
and factual knowledge were also studied. 

In the structural analysis dimension, the structural units were examined in which the given key concept or 
knowledge related to it occurred (Devetak & Vogrinc, 2013). The structural units studied were the following: 1) 
Main text (mandatory concepts, knowledge defined by the curriculum, which composes the main unit of teaching 
and learning in textbooks); 2) Reading (additional to the main text and can be clearly distinguished from it, which 
serves to supplement and motivate the main text); 3) Complementary text (non-compulsory units containing minor 
additional information formally separated as part of the main text); 4) Figure, image (its purpose is to illustrate the 
content of the main text, to help understanding, to motivate). This categorization of the structure was justified 
by the fact that the learning process can be influenced by the occurrence of concepts and knowledge in a given 
type of textbook structure element. What students are sure to read and learn is the main text of the curriculum. If a 
given concept or knowledge is present only in reading, in additional text or only in the form of a figure or picture, 
the students do not pay special attention to it, they will overlook it if the teacher does not specifically draw at- 
tention to it. Presumably, the most effective way of learning is when students encounter a given knowledge in as 
many structural elements as possible at the same time. Based on these aspects, all the occurrences were studied 
by grades and subjects and the co-occurrence of the key concepts in the given structural units was also analysed. 

All occurrences were given 1 point but if the key concept was missing 0 point was given. As a result, dichoto- 
mous variables were used in all cases. 

Analyses were carried out by two scientists (who studied all of the textbooks, involved in the research) and 10 
teachers (two teachers for each subject who analysed only the textbooks of their own subject). The results of the 
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two scientists and the two teachers were compared by subjects. As a result, the similarity of the data (considered 
categorized items) was studied by several coders. Considering correspondence, the Fleiss’ kappa value was .94 - .97 
(Natural history: .94; Biology: .95; Physics: .97; Geography: .96; Chemistry: .97). Key concept occurrences where two 
or more differences were found between the data of the four analysers were deleted. 


Data Analysis 


For the occurrence of key concepts (with the exception of co-occurrence according to the given criteria, where 
the Valid Percent values of the frequency test were used), relative frequency values were given. The occurrence 
of key concepts was given 1 or 0 points on a dichotomous scale (based on whether it occurred according to the 
aspect under consideration). 

Cochrans’Q test was performed: 1) in the evaluation in each dimension, to analyze the distribution by key 
concept according to the given aspect for the total sample (when searching for key concepts, all cases studied 
in all textbooks, including if there was no occurrence in the given textbook); 2) in the content, pedagogical and 
structural dimensions, to analyze the distribution of key concepts ( co-occurrences of key concepts) according to 
more than two aspects within a grade or subject. 

The Chi-Square test of the Crosstabs option in Descriptive Statistics was used: 1) to examine the distribution 
of each key concept (each key concept separately) by grades and subjects; 2) to analyze the occurrence of key 
concepts (key concepts together) in specific aspects of a given dimension by grades and subjects. 

Wilcoxon test was used to evaluate: 1) the content, pedagogical, and structural dimensions according to two 
criteria for the total number of cases that take into account actual occurrences; 2) key concept distributions within 
dimensions according to two aspects within grades and subjects. 

The Frequencies option of Descriptive statistics was used to examine the percentage of key concept (Valid 
Percent) occurrences, depending on the dimension, once (in one aspect, e.g., as a definition or circumscription 
within a conceptual dimension) twice (e.g., as both definition and circumscription) or in multiple aspects (triple or 
quadruple occurrence). When analysing these single or multiple occurrences, only differences within grades and 
subjects were examined, for which the Chi-Square test was used. Statistical analysis of the data was carried out 
using SPSS version 26.0. Effect size (d) measurements were made for each statistical test. 


Research Results 
Conceptual Content Analysis from a Professional Aspect 


The first question of the research concerned conceptual content analysis from a professional aspect. For 
the answer, the occurrence of key concepts related to EA specified by the National Curriculum (2012 & 2020) was 
examined in the studied textbooks. 

First, the relative frequency (see Table 1) of each key concept was determined for all studied textbooks and 
occurrence numbers (N = 621) (total sample) (see Table 2). 


Table 2 
Relative Frequency and Standard Deviation of Key Concepts Related to Energy Awareness in the Total Sample and for the 
Number of Occurrences (N = 621). 


Renewable Energy Heatin Power Conscious Energy Energy Energy 
energy saving g station energy use consumption efficiency crisis 
M 0.75 0.38 0.52 0.66 0.40 0.42 0.12 0.63 
SD 0.43 0.49 0.50 0.47 0.49 0.49 0.33 0.48 


The total number of cases tested for all occurrences in the total textbook sample was N = 621. The difference 
between the relative frequency values of key concepts was significant (Cochrans’Q = 65.48; df = 7; p < .001;d= 
0.686). Based on the d value, the distribution in textbooks showed an intermediate effect. The concepts of renew- 
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able energy, energy crisis and power station appeared in the largest proportions, followed by heating, energy 
consumption, conscious energy use, then energy saving and energy efficiency. 
The distribution of the above key concepts by grades was also studied (see Table 3). 


Table 3 


Relative Frequency of Key Concepts Related to Energy Awareness According to Grades (N = 621) 


/Print/ 


can Pi Ea gy Fa Sms oy oa gp 
1; 33 33 .00 .00 .00 33 .00 .00 5.71 7 573 1.117 large 
2. .00 33 .00 .00 33 .00 .00 .00 8.11 7 322 1.188 large 
3. 33 33 33 33 33 33 33 .00 2.66 7 914 0,706 intermediate 
4. .00 1.00 .00 33 1.00 .00 33 .00 8.00 7 333 0.752 _ intermediate 
5. 86 50 25 Gi .00 .00 .00 00 19.03 7 .008 2.180 large 
6. 1.00 1.00 1.00 1.00 1.00 .00 .00 00 22.00 7 .003 2.625 large 
7. 95 50 63 84 55 66 13 56 25.40 7 001 1.151 large 
8. 81 10 63 89 40 63 M1 44 25.86 7 .001 1.114 large 
9. .90 25 15 81 33 22 .00 44 3.,30 7 <.001 2.106 large 
10. 13 .08 54 42 40 63 28 55 2.13 7 <.001 1.021 large 
11. 60 25 .00 50 .00 .00 .00 50 1.06 7 185 1.353 large 
12. 1.00 50 AQ .66 1.00 50 50 50 5.91 7 541 1.474 large 
oa 32.18 13.68 21.05 39.25 16.17 21.20 9.69 13.29 
df 11 11 11 11 11 11 11 11 
p < 001 251 033 < .001 135 031 558 215 

Effect 1.237 1.108 1.353 1.359 1.148 1.359 0.885 1.003 

size(d) large —_ large large large large large large —large 


Taking the key concepts together, there was a significant difference in the distribution by grade (7 = 86.42; 
df= 11; p < .001; d= 0.804). Based on the d value, the impact of grades was large in terms of the relative frequency 
of the key concepts. Analysing the distributions in detail (see Table 3), there were significant differences in the case 
of renewable energy, heating, power station and energy consumption according to grade. The concept of renew- 
able energy and the knowledge associated with it were most prevalent in grades twelve, six, seven, nine, and then 
five and eight. The lowest relative frequency values were observed in grades 1-4. According to the occurrence of 
key concepts, two large groups could be distinguished. One was grades 1-4, where zero or low relative frequency 
values were typical, and the other was grades 5-12, where these values were much higher. 

In the case of the key concept’heating; similar two groups could be distinguished. Grades 1-4 with low or zero 
relative frequency values and grades 5-12 (except for grade 11, where there is zero occurrence) with higher values. 
The greatest occurrence of the latter was observed in grade six, and then in grades nine, seven, eight, ten and twelve. 

In the case of the key concept ‘power station, the same two groups with similar relative frequency values 
were observed as in the case of the previous two concepts. This concept was present in all the textbooks studied 
in grade six. Then grades eight, seven, and nine followed, between which there was no significant difference 
(7 = 1.55; df= 2; p = .460), and grades twelve, eleven, and ten brought up the rear. In the case of ‘energy consump- 
tion, the presence of this key concept was negligible in grades 1-6, and in grades nine and eleven, while it was 
present with a greater proportion in the other grades. 

Analyzing distributions within each grade, students in grades 1-2 encountered knowledge related to ‘renew- 
able energy’ ‘energy saving; ‘conscious energy use’ and ‘energy consumption’ while in grade three all key concepts 
appeared except for’energy crisis: In grade four, the concepts ‘renewable energy’and ‘heating’ were missing. Grades 
five and six were significantly (p < .05) lacking the knowledge of ‘energy crisis; ‘energy consumption, and ‘energy 
efficiency’, while in other grades, except for grade eleven, all key concepts occurred. In these latter grades, the 
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knowledge of ‘renewable energy; ‘power station, and ‘energy consumption’ occurred in the highest proportion, 
while ‘energy efficiency’ and ‘energy crisis’ have the smallest proportion. 

Overall, the concept ‘energy saving’ was encountered in all grades, followed by the concepts ‘renewable 
energy’ and ‘power station, while the lowest occurrence was shown by ‘energy efficiency’ and ‘energy crisis, which 
was especially typical in grades 7-12, however, with values below .56 relative frequency there as well. 

The occurrence of key concepts related to EA was examined according to subjects as well (see Table 4). 


Table 4 


Relative Frequency of Key Concepts Related to Energy Awareness According to Subjects (N = 621) 


Subject Renewable Energy Heating Power Conscious Energy Energy Energy Fe df p d Effect size 
energy saving station  energyuse consumption efficiency crisis (a) 

Newel CS 50 33 06 12 00 2335 7 001 0924 — large 
history 
Biology 42 23 38 22 30 ee 08 29 546 7 604 0473 — small 

. < 
Physics 84 545 88 30 50 00 7 351 7 gy yang large 
Geogr 400 1.002 94 66 15 33 66 650 7) (0.328 large 
phy 001 
nn eC 70 66 22 50 A 2 945 7 419 0.721 intermedi 
istry ate 
3740 218 1061 40.54 687 18.18 5.85 23.63 

af 4 4 4 4 4 4 4 4 

p <.001 702 031 <.001 143 001 210 © <.001 

d 1.387 0.397 0.866 1.390 0.685 1219 0629 1.493 

ra large small __large lage Meret stg interme isa 
size (d) g g g ate 9 diate g 


Taking the key concepts together, there was also a significant difference in the distribution by subject 
(7 = 114.92; df= 4; p < .001; d = 0.953). Value d again indicated here that the occurrence of key concepts was also 
strongly influenced by subjects. When analysing each key concept by subject, significant differences were detected 
in the case of the concepts ‘renewable energy; ‘heating; ‘power station; ‘energy consumption’ and ‘energy crisis’ 
according to each subject (see Table 4). Apart from the concepts ‘heating’ and ‘energy crisis, the highest relative 
frequency values of the key concepts could be observed in the textbooks of the geography subject, followed by 
physics, then by the values of chemistry and biology textbooks. In the case of key concept distributions indicating 
significant differences, the influence of subjects showed a large effect (see Table 3). 

Considering the relative frequency values of key concepts within given subjects, significant differences 
could be found in the textbooks of the subjects of Natural history, Physics, and Geography (see Table 4). In the 
textbooks of natural history, the concepts ‘renewable energy’, ‘power station’ and ‘energy saving’ were encoun- 
tered the most, while the concepts ‘energy efficiency’ ‘energy consumption’ and ‘energy crisis’ occurred in the 
smallest number. In physics, ‘conscious energy use’ had a lower relative frequency, while ‘energy efficiency’ was 
completely absent from physics textbooks. The other key concepts showed a relative frequency value of 0.50 
or more. Textbooks of geography showed a dominant role in the occurrence of key concepts. For example, 
knowledge of ‘renewable energy’ and ‘energy saving’ was present in all studied geography textbooks, but the 
concept ‘power station’ also had a high relative frequency (M = 0.94). ‘Energy efficiency’ was discussed in the 
lowest proportion in geography textbooks (M = 0.33), but the relative frequency of all other concepts was above 
0.60. In the textbooks of the subject of chemistry and biology, all concepts could be found. In biology, each 
occurrence was below 0.50. In chemistry, ‘renewable energy’, ‘heating, and ‘power station’ showed occurrence 
values higher than 0.50. With the exception of biology and chemistry, the occurrence of key concepts within 
subjects also showed a large influence which indicated a significant influence of subjects on the occurrence 
of key concepts. 
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Overall, it can be observed (see Table 4) that all the key concepts occurred in all subjects except for ‘energy 
crisis’ and ‘energy efficiency’ The concept and knowledge of ‘energy crisis’ were absent from the textbooks of the 
natural history subject, while ‘energy efficiency’ was missing from physics textbooks. Geography textbooks domi- 
nated in terms of occurrences. 

In the second part of the first research question (1b), it was analysed how key concepts relating to energy 
awareness were defined in a given textbook, as a specific definition, circumscription, or both. In this analysis, only 
those cases were examined where there was an actual occurrence, while textbook cases where the occurrence of 
key concepts was not observed were eliminated. Thus, out of a total of 621 cases, 320 cases could be investigated 
further. The analysis here was no longer carried out by key concepts, the co-occurrence of all concepts was inves- 
tigated according to the method of definition. 

For the 320 cases, the relative frequency of definition (M = 0.68; SD = 0.46) and circumscribed (M = 0.34; SD = 
0.47) type descriptions showed a significant difference (Z = - 6.174; p < .001; d= 0.733) with intermediate impact 
size. The obtained result clearly shows that the key concepts were described more in the form of a specific defini- 
tion regarding all occurrences. 

The distribution of key concepts by grades and subjects was examined here as well. Studying the definition 
description, the effect of grades (y?= 31.60; df=11; p = .001; d= 0.636) on the textbook occurrence of key concepts 
overall was intermediate and significant. 

In the distribution of definition type descriptions by grade, it was found that, except for grade four (M = 0.33), 
the relative frequency values were above 0.50 (see Table 5). Outstanding is grade eleven, where this value was 1.00, 
i.e., all key concepts were present in the form of definitions. 


Table 5 
Relative Frequency of Key Concepts Related to Energy Awareness According to the Method of Identification by Grades (N =320) 


Grade Definition Circumscription Z p Effect size (d) 

1. 40 60 -.45 655 0.411 small 
2. 50 87 “1.34 .180 1,076 large 
3. 50 83 -1.00 317 0.894 large 
4. 33 1.00 -1.00 317 0.894 large 
5. 17 23 -1.94 .052 3.489 large 
6. 87 13 -2.84 005 2.864 large 
7. 62 43 “1.72 085 0.420 small 
8. 74 27 -3.97 < .001 1.078 large 
9. 84 16 474 < .001 1.819 large 
10. 54 AT -.59 558 0.137 small 
11. 1.00 .00 -2.23 025 0.547 intermediate 
12. 60 40 -0.63 527 0.407 small 
Xx? 31.60 31.66 
df 11 11 
p 001 001 

Effect size (c 0.636 0.663 

intermediate intermediate 


For the circumscriptions, the distribution by grade for 320 cases also showed a significant difference (y? = 
31.66; df=11; p =.001; d= 0.663) and indicated the intermediate effect of the grade on the distribution of key con- 
cepts in this type of description. Compared to the definition description, occurrence rates here were the opposite. 
Only a circumscription was found in grade four, while no circumscription was found at all in grade eleven. Relative 
frequency values were above 0.60 in grades 1-3, and below 0.50. in other grades. 

Considering the two types of description within the grades, circumscription dominated in grades 1-4, while 
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definition dominated in higher grades. This trend showed significant differences in favour of definitions in grades 
six, eight, nine, and twelve. 

The influence of subjects on the textbook occurrence of key concepts (7? = 4.40; df = 4; p = .354; d= 0.211) 
in the case of definition type description was small overall and this effect was not significant. A similar result was 
obtained for the circumscription type description (7? = 5.94; df= 4; p =.203; d= 0.275) (see Table 6). 


Table 6 
Relative Frequency of Key Terms Related to Energy Awareness According to the Type of Description by Subjects (N=320) 


Type of Natural F ; : 2 Effect size 
description history Geography Biology Physics Chemistry x df p (a) 
Definition 13 65 68 18 62 4.40 4 354 0.211 small 
chili 38 36 32 22 45 5.94 4 203 0.275 small 
Z 2.53 3.70 1.89 4.18 96 
p 011 <.001 .059 < 001 336 
0.848 0.600 0.765 1.365 0.365 
Effect size (d) 
large intermediate intermediate large small 


The picture here is quite clear. In the case of all subjects, the definition type description was more frequent 
(see Table 6). There was a significant difference between the relative frequency values of the two types of de- 
scription in the textbooks of natural history, physics, and geography subjects. This shift was best observed in the 
textbooks of natural history and physics subjects. In the case of the rest of the subjects, this value was around M = 
0.60. Analyzing the relative frequencies within the definition and circumscription type descriptions, there was no 
significant difference in the distribution of key concepts by subject in either case. Overall, definition-type descrip- 
tions dominated the above circumscriptions in terms of both grade and subject. 

Lastly in the content analysis, it was examined whether the key concepts occur with one or both types of de- 
scriptions simultaneously in the given textbook. For all cases of occurrence (N = 320), 307 cases were found where 
the occurrence of the concept was single (either definition or circumscription) and 13 cases where it was double 
(in both types of description) (y? = 273.80; df= 1; p < .001; d= 4.868, large effect). 

Studying co-occurrence by grade (y? = 100.89; df= 11; p <.001; d= 1.296), a strong influence of grades on single 
or double occurrences of key concepts was found. Only one type of description was typical in the occurrence of 
key concepts in grades five, six, nine, eleven, and two, while in the other grades, both single and double. There was 
a strong significant difference in grades seven (p < .001) and eight (p < .001) in favour of the single occurrence. In 
contrast, in grades 1-4 there was no significant difference, single and double occurrences had a similar proportion. 

Co-occurrence by subject (y? = 17.77; df= 4; p < .001; d=0.484) indicated a small effect size for subjects. In biol- 
ogy, physics, and chemistry there were only single occurrences, while in geography and natural history, both single 
and double occurrences were found, but the single occurrence was more typical in the case of the latter subject. 


Content Analysis from Pedagogical Aspect 


Question 2a of the examination of the pedagogical dimension concerned what didactic elements are used 
to deepen and apply knowledge related to EA in the textbooks studied, and what proportions can be observed 
based on this aspect in the distribution of key concepts between the individual didactic elements. 

Three typical didactic elements were found in textbooks: 1) textbook tasks that can be solved in class, 2) inde- 
pendent homework, and 3) tasks that require an internet application. Thus, during the evaluation, the key concept 
distribution according to these types of tasks was studied. The number of cases here was 115. This was obtained by 
subtracting from the number of actual occurrences in the textbooks (320 cases) the number of cases where there 
was no task for the given key concepts (205 cases). Similarly to content analysis, a distribution test by task type 
for the total number of cases (N= 115) was carried out and the distributions by grade and subject were analysed. 

In examining textbooks by grade, with the exception of grades seven, eight, nine and ten, the total number 
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of tasks in the other grades, taken together for the three types of tasks, was no more than seven per grade. When 
these small numbers of occurrences were studied in proportion to the total number of textbooks studied in that 
grade, it was found that they give a statistically meaningless result due to the small number of cases (for example, 
in grade one only one task was found in the four textbooks). The proportions observed in the distribution of key 
concepts by task type (in one, two or three task types together) were also studied. 

Occurrence values of key concepts for all (N = 115) cases (textbook task: M = 0.93; SD = 0.24; Independent 
homework: M = 0.67; SD = 0.46; Task requiring Internet use: M = 0.23; SD = 0.42) (Cochrans’Q = 103.73; df = 2; p < 
.001; d = 6.064) indicated a significant and strong impact on the influence of task types. 

In the case of textbook tasks, distribution by grades seven, eight, nine, and ten showed a significant differ- 
ence (7?= 1.28; df= 3; p = .733; d= 0.230). The impact of grades on the occurrence of key concepts in these tasks 
is small (see Table 7). In the course of the examination by grades, key concept distributions in relation to textbook 
tasks that could be evaluated statistically were found only in grades seven, eight, nine, and ten. So, in this case, the 
number of evaluable cases is 97 (see Table 7). 


Table 7 
Relative Frequency of Key Concepts Related to Energy Awareness According to Task Types by Grades (N = 97) 


Textbook Independent Task requiring 


Grade task homework the use of the ae : df p d ae 
Internet 
7. 1.00 38 16 21.29 2 < .001 7.057 large 
8. .96 16 23 29.84 2 < .001 6.930 large 
9. 1.00 13 15 25.12 2 <.001 7.310 large 
10. 0.96 86 31 25.81 2 < .001 5.688 large 
0 1.28 13.00 2.02 
df 3 3 3 
p 733 .005 566 
0.230 0.782 0.291 
Effect size (d/) 
small intermediate small 


A significant difference in the distribution of key concepts according to the grades in Table 6 was found only 
in the case of independent homework (y= 13.00; df = 3; p =.005; d = 0.785), where grade ten had the highest pro- 
portion (M = 0.83) of independent homework related to the application of key concepts, followed by grades eight 
(M = 0.76) and nine (M = 0.73). The lowest rate was found in grade seven (M = 0.38). 

The proportion of textbook tasks for deepening and applying key concepts was significant and the highest in 
all grades followed by independent homework and then tasks requiring internet use. The proportion of the latter 
was significantly less compared to the other two (v?= 2.02; df= 3; p = .566; d= 0.291). 

When analyzing the key concepts from a didactic point of view, in the analysis by subject, the textbooks for 
all grades were used, as opposed to the examination of the distribution by grade, because the number of cases 
for each subject was evaluable (in the case of natural history subject, e.g., the textbooks for grades 1-6 were con- 
sidered together, which gave an evaluable number of cases). Therefore, here the number of cases of occurrence 
to work with was again N= 115. 
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Table 8 


Relative Frequency of Key Concepts Related to Energy Awareness According to Task Types by Subjects (N =115) 


Pyne Rate ian Geography Biology Physics + Chemistry x df p a 
Textbook task 60 .98 1.00 .96 1,00 20.96 4 < .001 0.959 Large 
Independent homework 40 67 55 87 85 13.72 4 .008 0.747 Intermediate 
Task requiring internet 30 18 44 19 42 4.63 4 327 0.415 Small 
Cochran's Q 1.55 59.91 6.00 39.46 5,20 
df 2 2 2 2 2 
p 459 < .001 < .001 < .001 074 
Effect size (c) 0.856 0.806 0.828 0.123 0.725 
large large large large large 


In the case of occurrence by subject, significant differences were found in the occurrence of key concepts 
by task type in the textbooks of geography, biology, and physics (see Table 8). Textbook tasks were found in the 
highest proportion followed by independent homework tasks, and by tasks requiring internet use in the case of 
all three subjects. Although there were no significant differences in the occurrence of key concepts by task type 
in the textbooks of natural history and chemistry subjects, a similar trend was observed in the distributions as in 
the case of the previous three subjects. 

Analysing each task type, the proportion of textbook tasks (y?= 20.96; df= 4; p <.001; d= 0.959) (see Table 8) 
was only smaller (M = 0.60) in the case of the natural history subject compared to the other subjects. Regarding 
independent homework (y?= 13.72; df= 4; p =.008; d= 0.747), physics (M = 0.87) and chemistry (M = 0.85) textbooks 
had the highest number of tasks compared to the textbooks of other subjects. The proportion of tasks requiring 
Internet use (y? = 4.63; df= 4; p =.327; d= 0.415) was small for all subjects. 

The distribution of key concepts by the three types of tasks was examined in the case of the didactic analysis 
as well. That is, whether the given key concept occurred in one, two, or all three of the textbook, homework, and 
Internet tasks. 

In the total sample (N =115), the key concept studied occurred only in one task type in 36 cases, in two task 
types in 61 cases and in three task types in 61 cases. In the case of co-occurrences, the effect of grades was found 
to be strong (y? = 25.29; df=11; p =.001; d= 1.080). In grades 1-6, single occurrence was dominant against double 
and triple occurrence (in grades 1-2 and 5-6 only single, in grade three single and double, in grade four single and 
triple), while in grades 7-12, co-occurrence in the three types of tasks was observed (double occurrence had the 
highest proportion, while triple occurrence had a smaller proportion). 

In the co-occurrence of key concepts according to didactic aspects (y? = 38. 68; df= 4; p =. 015; d=1.423) a 
significant difference was observed according to subjects and the subject effect was strong. In the case of each 
subject, single, double, and triple occurrences were found. In the case of natural history, the single occurrence (> 
80 %) was the most typical, and in the textbooks of the other subjects, the double occurrence was the most typi- 
cal. In summary, the occurrence of key concepts in the textbook tasks dominates both by grade and by subject, 
while tasks requiring internet use help the learning and application of key concepts related to energy awareness 
in the smallest proportion. 

In the following, it was examined whether a given key concept appeared in textbooks only as knowledge or 
as part of attitude shaping (question 2b). The values of the actual occurrences (N = 320) related to the total sample 
(Attitude shaping: M = 0.17; SD = 0.37; Knowledge: M = 0.99; SD = 0.08; Z = -15.52; p < .001; d =3.489) indicated a 
significant difference between the occurrences of the key concept at the level of factual knowledge and as part of 
attitude shaping, and the effect size was large. 
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Table 9 


Relative Frequency of Key Concepts Related to Energy Awareness According to Attitude-Shaping Aspects by Grades (N = 320) 


Grade Attitude shaping Knowledge Z p Effect size (d) 

ts 50 1.00 -1.00 < .001 2.000 large 
2. .00 1.00 -6.00 < 001 1.123 large 
3. 80 80 -0.00 1.000 0.000 no effect 
4. 1.00 1.00 -0.00 1.000 0.000 no effect 
5. 1.00 1.00 -0.00 1.000 0.000 no effect 
6. 92 1.00 -1.00 317 0.183 small 
7. 16 98 -7.28 < .001 3.891 large 
8. .06 1.00 -7.87 < .001 7.808 large 
9. .00 1.00 -7.00 < .001 9.123 large 
10. .05 1.00 -8.36 < .001 7.396 large 
ale 20 1.00 -2.00 046 2.828 large 
12. 00 1.00 -3.16 .002 10.665 large 
xv 12.73 15.08 
df 11 11 
p < 001 001 

Effect size (d/) 1.813 3.134 

large large 


The key concept distribution as part of attitude shaping according to grades (v7 = 12.73; df=11; p < .001; d 
= 1.813) indicated a significant and strong effect (see Table 8). The occurrence of key concepts as a textbook text 
for attitude shaping showed relative frequency values of .50 or higher in grades 4-5 (M= 1.00), six (M = 0.92), then 
three (M = 0.80) and one (M = 0.50). In other grades, this value was 0.20 or lower. 

The effect of grades on the occurrence of key concepts in textbook parts transferring knowledge (v7 = 15.08; 
df=11; p= .001; d= 3.134) was also significant and large. Relative frequency values of .80 or higher were observed 
in each grade. 

Regarding relative frequency values within grades, in grades three (M = 0.80), four and five (M = 1.00), key 
concepts occurred in equal proportions in textbook parts shaping knowledge and attitude. In all other grades, 
except for grade six (p = .317), the relative frequency values in textbook parts that transfer knowledge were sig- 
nificantly higher. 

The key concept distribution as part of attitude shaping by subjects (y? = 157.40; df= 4; p < .001; d= 1.967) 
indicated a significant and large effect (see Table 9). The relative frequency of key concepts in the attitude-shaping 
text was greatest in the textbooks of the natural history subject, while relative frequency values in the case of the 
rest of the subjects were 0.21 or smaller. 


Table 10 
Relative Frequency of Key Concepts Related to Energy Awareness According to the Aspects of Attitude Shaping by Subject 
(N= 320) 
Attitude shaping Natural 
aspects history Geography — Biology Physics Chemistry va df p Effect size (d) 
Attitude shaping 91 05 16 03 21 157.40 4 < .001 1.967 Large 
Knowledge 97 99 1.00 1.00 1.00 3.36 4 499 0.206 Small 
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Attitude shaping Netural 
aspects history Geography —_ Biology Physics Chemistry x df p Effect size (d) 


Z -1.00 -12.04 -4.58 -7.28 -4.69 
p 317 <.001 < 001 <.001 < 001 
0.348 7.248 4.567 9.291 3.826 

Effect size (d) 
Small Large Large Large Large 


Relative frequency in information-providing textbook sections by subject (y? = 3.36; df= 4; p =.499; d= 0.206) 
showed no significant difference, the effect of the subjects in this case was small. Relative frequency values in the 
textbooks of all subjects were 0.97 or greater. 

In the case of all subjects, key concept relative frequencies were significantly greater in textbook parts that 
provide knowledge except for the natural history subject in which there was no significant difference (p = .317) 
between occurrences in attitude-shaping and information-providing textbook parts. Overall, the vast majority 
of the textbooks studied discuss the studied key concepts at the factual knowledge level compared to textbook 
sections related to attitude shaping. 

With regard to the aspects of attitude shaping, the examination of joint occurrences in the attitude-shaping 
and information-providing sections was also carried out. We examined whether the given key concept occurs only 
in one or both types of textbook material. Considering the total number of cases studied (N = 320), there were 261 
single and 59 double occurrences. 

There was a significant difference between double and single occurrences by grades in the total sample 
(N=320) (v7 = 165.08 df= 11; p < .001; d= 2.064) and the effect of grades here was strong. In grades two and 7-12, 
a single occurrence was significantly characteristic, primarily as information-providing parts (> 85%). In grade one, 
the attitude-shaping and information-providing sections regarding the key concepts occurred in equal propor- 
tions (= 50%), while in grades 3-6, co-occurrence in attitude-shaping and information-providing sections was 
characteristic (double occurrence: > 60%). 

Examining single and double key concept occurrences by subject (vy? = 145.69; df= 4; p <.001; d= 1.828) there 
was also a significant difference between them, and a large effect of the subjects was detected. In natural history 
textbooks, the double occurrence ( > 80%) was characteristic, and in all other subjects, the single occurrence of 
key concepts ( > 78%), primarily as an information-providing textbook section was typical. 


Content Analysis from Structural Aspect 


In the course of the content analysis from the structural aspect, the differences in the occurrence of key con- 
cepts were examined in the core material, readings, complementary text of the textbooks or in the form of a figure 
or image with the evaluation of the distribution of key concepts among the structural units of textbooks (Research 
question 3). Studied cases concerned here again only the actual occurrences (N = 320). This analysis was pointless 
in those textbooks where the concept did not occur. 

Considering the total number of cases (N = 320), the occurrence of key concepts in the core material, reading, 
complementary text, or in the form of a figure or image (in core material: M = 0.84; SD = 0.35; in reading: M = 0.09; 
SD = 0.29; in complementary text: M = 0.09; SD = 0.28; figure, picture: M = 0.27; SD = 0.44) (Cochrans’Q = 270.07; df 
= 3; p < .001; d= 4.561) showed a significant difference verifying a large effect. 

Examining key concept occurrences in the core material by grade (7? = 26.17; df = 11; p =.006; d= 0.593) a 
significant difference was found indicating an intermediate effect of the grades (see Table 10). The occurrence of 
key concepts in the main text by grade shows relative frequencies greater than 0.50 in grades apart from 1-2 and 
three and eleven (see Table 11). In grade three, no key concept was encountered in the core material. 
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Table 11 


Relative Frequency of Key Concepts Related to Energy Awareness According to the Aspects of Structure Analysis by Grade (N= 320) 


Grade Main text Reading a ee a df p a 

1, 50 .00 .00 50 2.00 3 572 2.824 large 
2. 50 .00 .00 50 2.00 3 572 1.705 large 
3. .00 .00 .00 1.00 8,00 3 <.001 4.896 large 
4. 1.00 50 25 25 5,14 3 .162 3.322 large 
5. 80 07 33 46 1.02 3 .001 3.867 large 
6. 1.00 07 .00 53 33.69 3 <.001 4.063 large 
7. 19 MM 17 13 85.32 3 <.001 3.568 large 
8. .90 07 05 24 125.90 3 <.001 3.322 large 
9. 76 22 .08 45 48.39 3 <.001 17.426 large 
10. 92 .00 03 20 161.08 3 < .001 2.260 large 
11. 40 40 40 60 0.529 3 912 0.685 intermediate 
12. 90 10 .00 30 17.71 3 .001 5.603 large 
V7 26.17 35.89 30.76 31.56 
df 11 11 11 11 
p .006 <.001 001 001 

Effect size 0.593 0.710 0.648 0.659 
(0) intermediate interme- intermediate interme- 

diate diate 


Key concept occurrences in readings (y? = 35.89; df = 11; p < .001; d= 0.710) also showed significant differ- 
ences between grades and the effect of the grade here was also intermediate. In grades 1-3 and ten, the studied 
key concepts were not encountered in readings. Relative frequency values in readings were greatest in grade four 
(M = 0.50), while all other grades had relative frequency values of 0.40 or lower. 

In the complementary text again, significant differences could be detected in the relative frequency values 
of the key concepts by grade (y? = 30.76; df = 11; p =.001; d = 0.648) with intermediate effect size. There was only 
one outlier relative frequency value in grade eleven, but it was not high either (M = 0.40). In grades 1-3, no key 
concept was present in complementary texts. In the other grades, relative frequency values of 0.33 (fifth grade) 
or lower were detected. 

In the case of occurrence in the form of figures or pictures, there were significant differences by grade (y? = 
31.56; df = 11; p = .001; d= 0.659). The effect of grades, in this case, was also intermediate. Interestingly, in grade 
three, key concepts were encountered only in the form of figures, pictures. The relative frequency value was the 
highest in grade eleven (M = 0.60), followed by grade six (M = 0.53) and grades 1-2 (M = 0.50). In other grades, 
relative frequency values lower than 0.50 were found. 

Considering analysis within the grades, in grades 1-3, the occurrence of key concepts in the main text and in 
the form of a figure or image was typical (in grade three, only figure and picture occurrence could be observed). 
In the other grades, the occurrence in the main text dominated, compared to the occurrence in reading, comple- 
mentary text, figures, and pictures. Grade eleven was the exception, where the occurrence in figures and pictures 
dominated (M = 0.60), the other three had relative frequency values of 0.40. 

The occurrence of key concepts in the main text showed no significant difference by subject (y? = 4.42; df= 
4; p =.351; d= 0.238). The impact of the subjects, in this case, was small (see Table 11). The highest proportion of 
occurrences in the main text was detected in the natural history textbooks (M = 0.90), but for the other subjects 
the relative frequency value was less than 0.70 (see Table 12). 
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Table 12 


Relative Frequency of Key Concepts Related to Energy Awareness According to the Aspects of Structure Analysis by Subject 
(N= 320) 


Structural aspects fae Geography Biology Physics Chemistry xr df p Effect size (d) 
Main text 90 0.82 0.80 0.81 0.70 4.42 4 351 0.238 small 
Reading 10 0.04 0.04 0.29 0.13 2713 4 <.001 0.614 intermediate 
Complementary text 13 0.07 0.08 0.07 0.23 7.85 4 097 0.319 small 
Figure, picture 45 0.25 0.20 0.38 0.09 15.39 4 .004 0.453 small 
Cochran's Q 6.33 156.34 36.94 61.31 2.65 
df 3 3 3 3 3 
p < .001 < .001 <.001 < 001 < .001 
Effect size (d) 3.234 7.691 2.231 3.345 7.213 
large large large large large 


There was a significant difference in the distribution of key concepts occurring in readings by subject (vy? = 
27.13; df= 4; p < .001; d= 0.614), the subject effect is intermediate. Relative frequency was the highest in the case 
of physics (M = 0.29), in the readings of the textbooks of other subjects this value was 0.13 (chemistry) or lower. 

There was no significant difference in the occurrence of key concepts in complementary texts by subject and 
the subject effect was small (y? = 7.85; df = 4; p = .097; d = 0.319). Key concepts occurred most frequently in the 
complementary texts of chemistry textbooks (M = 0.23). In the case of other subjects, this occurrence showed a 
relative frequency of 0.13 (natural history) or smaller. 

The difference between the relative frequencies of key concepts in the form of figures and pictures by the 
subject was significant, but the subject effect was small (y? = 15.39; df= 4; p =.004; d= 0.453). Key concepts occurred 
in figures and pictures most frequently in natural history (M = .45), and physics textbooks (M = 0.38), while in the 
case of other subjects, relative frequency values of 0.25 (geography) or lower were detected. 

Key concepts occurred significantly in the greatest proportion in the main text in all subjects, followed by 
figures, pictures, and then in readings and complementary text. Overall, both the analysis by grade and by subject 
revealed that most of the knowledge related to EA occurred in the main text which contained the curriculum re- 
quirements, and then in the accompanying figures and pictures. The occurrences in readings and in complementary 
texts were smaller compared to the previous two. 

Co-occurrence was studied in the course of the structural analysis as well. The relative frequency of key con- 
cepts was analysed in one, two, three, or all four structural elements, i.e., the main text, readings, complementary 
texts, figures, and pictures. Considering the total number of cases (N = 320), 231 single, 64 double, 11 triple and 
4 quadruple occurrences were found. 

Studying the differences in co-occurrence by grade in the structural analysis (y?= 111. 91; df= 11; p <.001;d 
= 1.459) significant differences were found. The effect of grades on co-occurrence was large. Single occurrences 
were not found only in grades 2-3. The double occurrence was also absent in two grades, in the textbooks of grades 
one and three. The absence of triple occurrence was detected in grades one, two, seven and ten. The quadruple 
occurrence was only found in grades 7-9. 

Considering the structural analysis, significant differences were found in the valid per cent values in the dis- 
tribution of the co-occurrence of key concepts by subject (y?= 40.32; df= 4; p = .001; d= 0.800), and the subject 
effect was large. In the textbooks of all subjects, a single occurrence was dominant (in the main text), followed by 
double, triple, and quadruple occurrences. In the textbooks of geography and physics, all cases were detected, 
while in biology and chemistry, only single and double occurrences were detected. In natural history, only the 
quadruple occurrence was not found. 
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Discussion 


The aim of the study is to present textbook research that is now one of the most topical environmental edu- 
cation tasks in the period of the global energy crisis, exploring the potential of educating students on EA in 67 
Hungarian primary and secondary science textbooks (grades 1-12, 5 subjects: natural history, geography, biology, 
physics, and chemistry). In the field of environmental education, including EA education, as part of the attitude, it is 
important what kind of energy awareness knowledge students have, whether they can behave energy-consciously, 
and what emotions and beliefs they have about EA. The question is to what extent textbooks, including textbooks 
on science subjects, can enhance the threefold function of EA education mentioned here. 

Textbooks are important aids of learning, which reflect and thus provide an opportunity to achieve the learn- 
ing and teaching goals set out in the curriculum requirements (including the development of various student 
competencies) through the communication of the teacher. Consequently, their content, structure and pedagogical 
characteristics have a fundamental impact on the process of teaching and learning, in terms of the knowledge, 
abilities and attitudes to be developed (Chambliss & Calfee, 1998). The textbook analysis model, which emphasizes 
the constructive learning process and was chosen as the theoretical basis for our textbook research, devotes an 
important role to textbooks in the shaping of the knowledge, abilities and attitudes created as a result of learn- 
ing. According to the constructive model, one of the main aspects of textbook analysis is the extent to which the 
textbook helps to provide a student-centred teaching. For this, it is essential that the content of the textbook is 
understandable, concise, motivating for the student, contains tasks for deepening and applying knowledge, builds 
on the emotional factors of the students, presents the relationship between the knowledge acquired in school 
and everyday life, and encourages independent learning. They are all textbook functions, if implemented, the 
development of an attitude towards EA can also be ensured. 

As a meeting point between the main aspects of the textbook analysis model with constructive learning in 
mind (knowledge, ability and attitude) and the individual elements of the attitude towards EA (knowledge about 
EA, behaviour, emotions), quantitative conceptual content analysis was applied in three dimensions: 1) Professional 
content dimension: in which first the occurrence of the key concepts and knowledge related to energy awareness 
specified in the curriculum requirements (‘renewable energy; ‘energy saving; ‘heating; ‘power station’ ‘conscious 
energy use; ‘energy crisis, ‘energy efficiency, and ‘energy consumption’) was analysed in the studied science text- 
books. If these concepts occur in the textbooks, in what ways do they help students to understand and see the 
essence? Whether they occur in the form of short definitions or circumscribed text containing descriptions, or 
explanations. In the case of the occurring key concepts, their distribution between these forms was also studied. 
2) In the course of the content analysis from the pedagogical aspect, first the typical elements for the deepening 
and application of key concepts related to EA were identified in the studied textbooks. For completing this didactic 
task, studying textbook tasks, independent homework tasks and tasks requiring the use of the Internet seemed 
suitable. Thus, the distribution of key concepts related to EA according to these tasks was analysed here as well. In 
the second part of the pedagogical analysis, the occurrence of key concepts clearly related to EA was examined 
in the attitude shaping (behaviour, emotions, beliefs, attitude shaping) texts. It was examined whether the key 
concept occurred as a knowledge or attitude element. 3) In the course of the content analysis from the structural 
aspect, it was assumed that it was not irrelevant for a child, regarding learning, that the knowledge to be learned 
occurred in the main text containing the teaching material, or in motivating readings containing curiosities, or 
in images in figures that provide additional information, help understanding or motivate. Therefore, the relative 
frequency of the key concepts related to EA in the main textbook text, readings, supplementary texts and figures, 
images was analysed here. 

The occurrence of key concepts was examined first in the total sample in all dimensions, regardless of grades 
and subjects. Then occurrences were analysed by grade and subject. Finally, it was examined in each dimension 
that the key concepts occurred in how many of them. The reason for the latter analysis was that the more the given 
knowledge occurred, the deeper the created knowledge of the students about it would be. 

In the course of the content analysis from the professional aspect, it was found that key concepts related to EA 
were not present at all in 18 of the 67 textbooks studied. This may also be explained by the fact that the concepts 
and knowledge studied were not related to the content of the given textbooks. The concepts of ‘renewable energy’, 
‘energy crisis, and ‘power station’ occurred in the largest proportion in the total sample (N= 62, including multiple 
occurrences of a given key concept in different aspects in different topics of a given textbook) followed by the 
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concepts of ‘heating; ‘energy consumption; ‘conscious energy use; and then ‘energy saving’ and ‘energy efficiency’ 
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Breaking down the occurrence of key concepts by grade, two large groups could be distinguished. One group 
included grades 1-4, where zero or small relative frequency values were typical, and the other group included 
grades 5-12, where these values were much greater. In grades 1-4, the concepts of ‘renewable energy; ‘energy 
saving; ‘heating; and ‘power station’ were encountered, and then, towards grade 12 more and more key concepts 
related to EA occurred. In grades 1-4, the concepts that actually occurred were described mainly in a circumscribed 
form. In many cases, it was not even the key concept itself that was included in these descriptions, but another 
knowledge or concept within the scope of the key concept (e.g., the concept of ‘renewable energy’ in the text- 
books for grades 1-2 grade occurred as wind or water, which we can use to generate electricity as well). These were 
correctly conceptualized at an age-appropriate level of intellectual development, mostly in a childish language. 
However, from grade five increasingly abstract, short, definitions of key concepts were found in greater propor- 
tions, which can make their understanding and learning difficult (Adamik et al. 2004; Nagy & Varga, 2014), if there 
was no explanatory text, picture, figure, task, or teaching method to help understanding. This entails maintaining 
a definitive knowledge at a factual knowledge level in most grades, which is not favourable for the development 
of behaviour and emotional elements related to EA. 

When studying the occurrence of key concepts by subject, it was found in the total sample (N=621) that the 
key concepts occur in all subjects except for the concepts of ‘energy crisis’ and ‘energy efficiency’ The concept and 
knowledge of ‘energy crisis’ were absent from the textbooks of the natural history subject, while ‘energy efficiency’ 
was missing from the textbooks of physics. In line with previous studies (Konya, 2018b; Revakné, 2018), geography 
textbooks are dominant in terms of occurrences, followed by physics, chemistry, and biology. This result should be 
treated with care. In this case, it has to be taken into account that the number of topics to which EA can be related 
is different in the given subject textbooks (Konya, 2018). Furthermore, the question is whether textbooks always 
take advantage of mentioning and teaching the aspects of EA when discussing a given topic. A deeper exploration 
of this could be the subject of another research. Considering the description of the occurring concepts, a similar 
result to those of the grades can be seen. That is, definitions played a major role in the textbooks of all subjects, 
which may result in that information getting stuck at the factual knowledge level regarding EA attitude shaping. 

When the occurrence of key concepts was analysed, it was revealed that a single occurrence predominates 
(307 cases were either definition or circumscription and 13 cases were occurrences in both modes of description). 
Key concepts found in similar proportions in one or both forms were typical only in grades 1-3. The vast majority 
of single occurrences (grades 5-12) were in the form of definitions, which, in line with previous research (Coker et 
al., 2010), suggests a rather definitive nature of knowledge related to EA in the textbooks studied. 

In the didactic relations of the content analysis from the pedagogical aspect, the proportion of concept 
distribution between textbook tasks, independent homework and tasks requiring the use of the Internet was ex- 
amined in the case of actual key concept occurrences. Considering the total sample (only N=115 due to the actual 
occurrences in the tasks), in most cases the knowledge related to EA was deepened and applied in the textbook 
tasks, followed by the independent homework tasks, and finally in the tasks requiring internet use. The analysis 
by grade and subject yielded similar results. If we consider the fact that only 115 of the actual 320 occurrences in 
the 67 textbooks were observed in these tasks, this represents almost a third of the deepening and application of 
the key concepts that actually occur related to EA. Again, it should be taken into account that the content related 
to EA makes what proportion of the content of that subject. Regarding the textbook occurrence of key concepts, 
this one-third application is small and does not help to develop the attitude related to EA. Nor is it the case that 
only grades 7-10 had a key concept in each of the three types of tasks, and in the other grades only one of the 
three. Further, deeper analysis is required to identify the form of work (e.g., cooperative work, project, etc.), method 
(e.g., observation, experiment), thinking operation (e.g., problem solving, inductive thinking, divergent thinking, 
etc.) required by the solution of the above task types, all of which can be important components of EA education. 

In the section on attitude shaping in the content analysis from the pedagogical aspect, in accordance with 
previous research results (Chao.et al., 2017; Revakné et al. 2018), it was found that the vast majority of textbooks 
provided factual knowledge related to EA both in terms of all actual key concept occurrences and in terms of 
analysis by grade and subject. This is certainly a fact to consider in the future for textbook writers, partly because 
they only partially meet the requirements of the curriculum related to EA in terms of attitude development, and 
also because the development of a correct attitude related to EA is thus helped less by textbooks. 

The content analysis from the structural aspect focused on how the occurring key concepts were divided 
between the main text of textbooks, readings, complementary text and figures, and pictures. The examination of 
these four aspects of structural analysis was partly justified, on the one hand, by the fact that these were typically 
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the structural units that occurred almost everywhere in the structure of textbooks, and on the other hand, by the 
assumption that it does matter in what structural unit a particular knowledge occurred. The main text is the unit ina 
textbook which contains the most important information and curriculum requirements for learning, both students 
and teachers pay more attention to it, learn from it, and teach it. Apart from the main text students mainly focus 
on the explanatory, information-providing and motivating figures and images belonging to the main text. It is a 
common experience that students and their teachers often overlook readings and complementary texts, despite 
the fact that they often contain interesting information, not considering it important. However, they sometimes 
provide knowledge and complementary information to the content of the main text that help to understand it, give 
examples of the connection of the learned knowledge with everyday life, its practical application or even contain 
attitude-shaping elements. They would therefore be important in the development of applicable knowledge and 
in the shaping of attitude as well. 

Present research confirms this experience. Most of the studied knowledge related to EA occurred in the main 
text, and then in the accompanying diagrams and images. The occurrence rate in readings and complementary texts 
was smaller compared to the previous two. However, regarding this result, it should be noted that in textbooks the 
ratio of readings and complementary texts is smaller compared to the main text, figures, and images. Not all main 
texts have readings or complementary texts. When examining multiple occurrences in structural units, it was also 
detected that in the textbooks of different subjects, a single occurrence was typical, predominantly in the main text. 


Conclusions and Implications 


This study presented the analysis of natural sciences textbooks in relation to energy awareness. The analysis was 
based on the content analysis method which bears the constructive learning process in mind. The applied method 
was quantitative content analysis. In the course of the conceptual content analysis, the occurrence frequency of 
the key concepts related to EA was examined from professional, pedagogical and structural aspects. The results 
show that concepts related to ER are present in the textbooks studied. Most of the concepts occur in definition- 
like texts and circumscriptions which better facilitate understanding are less frequent. Textbook elements shaping 
attitude and behaviour are less frequent than factual knowledge. Thus, natural sciences textbooks to be edited 
in the future shall include more explanations and descriptions better helping and motivating the understanding 
of concepts related to EA. It would be worth devoting separate lessons, pages to this topic. An increase in the 
volume of texts, visual elements, tasks (tasks requiring activities, observation, student examinations related to EA, 
modelling, projects, collaborations studying everyday experience, tasks facilitating ICT and processing internet 
sources) shaping attitude and behaviour related to EA shall occur. It will still be a question, of how teachers would 
transfer the content of such corrected textbooks towards the students. As this fundamentally determines how 
the textbook content transforms into knowledge and how attitude and behaviour related to EA are shaped, and 
what role textbooks can take in shaping knowledge and attitude compared to other school and social factors. This 
could be the topic of future research. 
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